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The theory of PBW properties of quadratic algebras, to which this paper aims to be 
a modest contribution, originates from the pioneering work of Drinfeld (see \Drll). In 
particular, as we learned after publication of EGy (to the embarrassment of two of us!), 
symplectic reflection algebras, as well as PBW theorems for them, were discovered by 
Drinfeld in the classical paper \Dr2\j 15 years before \EG)j (namely, they are a special case 
of degenerate affine Hecke algebras for a ßnite group G introduced in \Dr2tj . Section 4). 

It is our great pleasure to dedicate this paper to Vladimir Drinfeld on the occasion of 
his 50-th Birthday. 



1 Introduction 

In |Dr2j . Section 4, Drinfeld introduced the family of degenerate affine (=graded) Hecke 
algebras attached to any finite group G and its linear finite dimensional complex repre- 
sentation V. The goal of this paper is to dehne and study the "continuous" generalization 
of these algebras, in which the group G is a reductive algebraic group, and V is its al- 
gebraic representation. We call this generalization continuous Hecke algebras. They 
include continuous symplectic reflection algebras and continuous Cherednik al- 
gebras, which are generalizations of symplectic reflection algebras from |EG| and rational 
Cherednik algebras. 

A motivation for studying continuous Hecke algebras comes from the fact that their 
representation theory (which is yet to be developed) unifies the representation theories of 
real reductive groups, Drinfeld-Lusztig degenerate affine Hecke algebras, and symplectic 
reflection algebras of |EG| (in particular, rational Cherednik algebras). 



Let us briefly describe the contents of the paper. Let Gbea reductive algebraic group 
1 with an algebraic representation p : G — > GL(V). Let 0{G)* be the algebra, with respect 
to convolution, of algebraic distributions on G (see subsection 12.1(1 . This algebra has a 
natural ö(G)-module structure. 

Let k G (Ö(G)* ® /\ 2 V*) G . Following the idea of Dr2 , we dehne the algebra H K to be 
the quotient of TV xi Ö(G)* by the "main commutation relation" 

[x,y] = K(x,y), x,yeV, 

and W6 9X6 intcrcstcd. in fi.nd.ing thosc k for which H K is a hat deformation of Ho = 
SV xi Ö(G)* (the PBW property). As in |EG| . it turns out that flatness is equivalent to 
the Jacobi identity 

k(x, y)(z - z 9 ) + n(y, z)(x - x 9 ) + k(z, x){y - y 9 ) = 0, x, y, z G V, g G G. 

We call the algebras H K satisfying the PBW property (equivalently, the Jacobi identity) 
continuous Hecke algebras. 

One of the main problems treated in this paper is to classify continuous Hecke algebras, 
i.e. to find explicitly all distributions k satisfying the Jacobi identity. In the case when 
G is finite, the answer is given in |Dr2j (see also |RS| ). However, for an infinite G the 
Situation is more complicated and we don't have a complete answer; so we present some 
partial results. 

Let $ C G be the closed subscheme of G defined by the equation A 3 (l — g\y) = 0. Our 
first result is that if k satisfies the Jacobi identity then it is supported on <E>. 

Next, let 9 G <g> A 2 V*) G , and r G {ö{Kerp)* ® A 2 V*) G . We show that the 

distribution 

k(x, y) := t(x, y) + ö((l - g)x, (1 - g)y) 

satisfies the Jacobi identity and hence defines a continuous Hecke algebra. This is the 
general answer if the group G is finite. 

Another collection of examples comes from Symmetrie pairs (G, K) of reductive groups. 
In this case H K = j4(G,K), the algebra of distributions on G set-theoretically supported 
on K. 

Further, we consider the special case when the space V carries a G-invariant sym- 
plectic form to. In this case, let £ be the closed subscheme of G defined by the equation 
p o A 3 (l — g\v) = 0, where p : A 3 V — > V is the projection defined by oj. Let 

k(x, y) = u(x, y)t + - g)x, (1 - g)y)c, 

where t is a G-invariant distribution on Ker/3, and c is a G-invariant distribution on S. 
Here u>((l — g)x, (l — g)y)c demotes the produet of the regulär funetion u((l — g)x, (l — g)y) 
by the algebraic distribution c, using the ö(G)-module structure on 0(G)*. 
1 Unless otherwise speeified, all algebraic groups and Lie algebras in this paper are over C. 
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We show that k satisfies Jacobi identity, and call the corresponding algebras H K con- 
tinuous symplectic reflection algebras. If G is finite, and p is a faithful representation, 
they coincide with symplectic reflection algebras defined in EG . 

Now let us assume in addition that V = l) © l)*, where f) is a representation of G (and 
the form u is given by the natural pairing between I) and fj*). In this case we show that 
n satisfies Jacobi identity if n(y, x) = if y, x G f) or y, x G fj*, and 

*(y, x) = (y, x)t + (y, (1 - g)x)c, y G fy,x G fy*. 

where i is a G-invariant distribution on Kerp, and c is an invariant distribution on the 
closed subscheme f CG, "of complex reflections" , defined by the equation A 2 (l — g\^) = 0. 
We call algebras H K for such k continuous Cherednik algebras. If G is finite and p is 
a faithful representation, they coincide with the rational Cherednik algebras considered in 

One of our main results is that if f) is irreducible of real or complex type and faithful 
then the classes of continuous Hecke algebras, continuous symplectic reflection algebras, 
and continuous Cherednik algebras are the same. 

We consider in detail the examples G = GL n , O n , Sp2 n , and V = C 2n with the natural 
symplectic form. 2 In all these examples, any continuous Hecke algebra H K is a continuous 
symplectic reflection algebra, and we classify explicitly the possible distributions c. In 
particular, for G = O n we find that there is just one essential parameter, while for G = GL n 
and G = Sp2 n there are infinitely many. 

If the distribution k is set-theoretically supported at 1 (i.e. belongs to Uq, where g is 
the Lie algebra of G), then one can define the algebra Jt K to be the quotient of TV x Uq 
by the main commutation relation. It is a flat deformation of SV x Uq if and only if k 
satisfies the Jacobi identity; in this case we call H K an infinitesimal Hecke algebra. We 
write such n explicitly in the cases G = GL n , G = Sp2 n (in the case G = O n , there are no 
interesting algebras J%%). 

An interesting problem is to develop representation theory of the algebras H K and Jif K . 
In the case of G = SL2, V = C 2 , this was done in |Khj . but other examples (apart from 
finite G) remain mostly unexplored. We begin to tackle this problem by introducing the 
category ö and studying the Dunkl-Cherednik representation of continuous Cherednik 
algebras. 

Remark 1.1. Note that continuous representations of the algebra A(G, K) are nothing 
but Harish-Chandra modules for the pair (G,K). Thus, the representation theory of 
continuous Hecke algebras is a generalization of the representation theory of real reductive 
groups. 

2 The example G = Sp2 = SL2 was considered earlier in Kh . 
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At the end of the paper, we consider the case of wreath products G = S n x T n , where 
r is a reductive subgroup of SL2, and V = C 2n . We consider in detail the three cases 
in which T is infinite, and generalize the results of Montarani on the Classification of 
representations of H K irreducible over G (proved in jM] in the case when T is finite). 

Acknowledgments. The work of P.E. was partially supported by the NSF grant 
DMS-9988796. The work of V.G. was partially supported by the NSF grant DMS-0303465. 
The work of W.L.G. was partially supported by the NSF grant DMS-0401509. The work 
of P.E. and V.G. was partially supported by the CRDF grant RM1-2545-MO-03. 

2 Continuous Hecke algebras 

2.1 Algebraic distributions. Let X be an affine scheme of finite type over C. We 
shall denote by 0(X) the algebra of regulär functions on X. An algebraic distribution on 
X is an element in the dual space O(X)* of Ö(X). For c G 0(X)* and / G 0(X), we will 
denote the value c(/) by (c, /). 

The space Ö(X)* is naturally equipped with the weak (inverse limit) topology. Note 
also that Ö(X)* is a module over Ö(X): for any / G Ö(X) and /j, G Ö(X)* we can define 
the element fß = /j,fe O(X)* by (ffi,g) = {n,fg). 

Let Z be a closed subscheme of X, and write I(Z) for its defining ideal in Ö(X). We 
say that an algebraic distribution fi on X is supported on the scheme Z if /i annihilates 
I(Z). Clearly, the space of algebraic distributions on X supported on Z is naturally 
isomorphic to the space of algebraic distributions on Z. 

Now assume that Z is reduced. We say that [i G 0(X)* is scheme-theoretically (re- 
spectively, set-theoretically) supported on the set Z if ß annihilates I(Z) (respectively, 
some power of I{Z)). 

Example 2.1. For each point a G X, the delta function 5 a G O(X)* is defined by 
$a(f) '■= /(a) where / G O(X). It is scheme-theoretically supported at the point a, and 
its derivatives are set-theoretically supported at this point. 

Let G be a reductive algebraic group. Since 0(G) is a coalgebra, its dual space Ö{G)* 
is an algebra under convolution. The unit of this algebra is the delta function 8\ of the 
identity element 1 G G. 

Note that a continuous representation of the algebra Ö(G)* is the same thing as a 
locally finite G-module (i.e. a G-module which is a direct sum of finite dimensional 
algebraic representations of G). 

Suppose that G acts on X. Then G acts also on Ö{X) and Ö(X)*. We have 
0{X) = Q) v My®V, where V runs over the irreducible representations of G, and My are 
multiplicity Spaces. Thus, Ö(X)* = ]\ v M V ®V* . In particular, Ö(G)* = \\ v V®V* as a 
G x G-module. 
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Recall that the categorical quotient X/G is an affine scheme of finite type with Ö(X/G) = 
Ö{X) G . It is clear that the space (Ö(X)*) G of invariant algebraic distributions on X is 
naturally isomorphic to the space 0(X/G)* of algebraic distributions on the quotient X/G. 
We will denote this space by C(X). Clearly, if Z C X is a G-invariant closed subscheme, 
then C(Z) is naturally a subspace of C(X). 

2.2 Definition and properties of continuous Hecke algebras. Let G be a reductive 
algebraic group (not necessarily connected), and p : G — > GL(V) be a finite dimensional al- 
gebraic representation of G. The semi-direct product TV x 0(G)* is the algebra generated 
by x G V and /i G Ö(G)* with the relations 

M . x = . (v*,gx)ß for all x G V, p G Ö(G)*, (2.2) 

i 

where Uj is a basis of F and the dual basis of V* . (Here (v^,gx)ß denotes the product of 
the regulär function (v*,gx) by the distribution jj). Given a skew-symmetric G-equivariant 
C-bilinear pairing k : V x V — > Ö(G)* (where G acts on Ö(G)* by conjugation), let 
H K = H K (G) be the quotient of TV x O(G)* by the relations 

[x, y] = k(x, y) for all x, y G V. (2-3) 

Dehne a filtration on H K by setting deg(x) = 1, deg(/x) = for x G V, ß G Ö(G)*. Note 
that Ho = x Ö(G)*. There is a natural map Ho — > gr(H K ) which is a surjective graded 
algebra homomorphism. We say that the PBW property holds for H K if this morphism is 
an isomorphism. 

For x &V, let x 9 denote the result of the action of g on x. 

Theorem 2.4. The algebra H K has the PBW property if and only if k satisfies the Jacobi 
identity 

(z — z 9 )k(x, y) + (y — y 9 )n{z, x) + (x — x 9 )K(y, z) = for all x,y,z G V. (2.5) 

Proof. The algebra Ho is Koszul. Therefore, by the results of |B(t| . |B(IS] (see also |E(t| 
§2]), the PBW property is equivalent to the Jacobi identity 3 

[k(x, y), z] + [k(z, x), y] + [n(y, z), x] = for all x, y, z G V. (2-6) 
By we have 

[z,K(x,y)] = (z-z 9 )K(x,y) G (F®0(G))® 0(G) 0(G)* = V ® 0{G)*. 

Hence, the Jacobi identity (|2.6j) is equivalent to equation (|2.5|) . as needed. □ 

3 Strictly speaking, the result of IBGSI is stated in the Situation when the degree zero part of the algebra 
is finite dimensional semisimple; however, one can check that all arguments extend to the case when the 
degree zero part is the inverse limit of finite dimensional semisimple algebras, in particular when the degree 
zero part is Ö(G)* . 
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Definition 2.7. 7/H K has the PBW property (equivalently, k satisfies the Jacobi identity) 
then H K is called a continuous Hecke algebra. 

Let $cGbe the closed subscheme defined by the equation A 3 (l — g\v) = 0. Then the 
set of closed points of 3> is the set S of elements of G such that the rank of the Operator 
1 - g : V -» V is < 2. 

Proposition 2.8. If the PBW property holds for H K , then n{x,y) is supported on the 
scheme <3? for all x,y G V. (In particular, K,(x,y) is set-theoretically supported on S.) 

Proof. Here and below, we will use the following notation: for any v±, . . . , v^, x, y 6 V, we 
write 

(yx,...,v k \x,y) for ( Vl - v f) A ---A(v k -v 9 k )K(x,y) G {A k V®ö(G)) ® (G)0(G)* = A k V®ö(G)*. 
Thus, we can rewrite the condition ()2.5I) as 

(z\x, y) + (x\y, z) + (y\z, x) = for all x,y, z G F. (2-9) 
To prove the proposition, we will need two lemmas. 
Lemma 2.10. If \2. y\) holds, then (z, u\x, y) = (x, y\z, u) for all x, y,z,u G V . 
Proof. We have 

(u,z\x,y) = -(u,x\y,z) - (u,y\z,x), 
(z,u\x,y) = -(z,x\y,u) - (z,y\u,x). 

Subtracting the first equation from the second equation, and using (|2.9|) . we get 

2(z,u\x,y) =((x,z\y,u) + (x,u\z,y)) + ((y,z\u,x) + (y,u\x,z)) 
=(x,y\z,u) + (y,x\u,z) 
=2(x,y\z,u). 

□ 

Lemma 2.11. If \2. ,91) holds, then (z, u, v\x, y) = for all x, y, z,u,v G V. 
Proof. By Lemma 12. IUI we have 

(v,z,u\x,y) = (v,x,y\z,u), 
(u,v,z\x,y) = (u,x,y\v,z), 
(z,u,v\x,y) = (z,x,y\u,v). 

Adding these three equations, we get 

3(z, u,v\x, y) = (x, y, v\z, u) + (x, y, u\v, z) + (x, y, z\u, v) = 0. 

□ 
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Now we prove Proposition 12.81 The PBW property is equivalent to condition (|2.9j) . 
Hence, the proposition is immediate from Lemma 12.111 □ 



2.3 Examples of continuous Hecke algebras. Let 9 := (ö($)* <g> A 2 V*) G , and 
00 C be the subspace of those 9 G for which 9((1 — g)x, (1 — g)y) = for any 
x, y G V. Let 6 G 0/©o, and let r G (O(Kerp)* gl A 2 V*) G . Dehne the distribution k by 

y) = r(x, y) + - g)x, (1 - g)y). (2.12) 

Theorem 2.13. The algebra H K for such k has the PBW property. 

Proof. Since (u — u 9 )t = 0, the Jacobi identity reduces to the relation 

(z - z 3 )6(x -x 9 ,y- y 9 ) + (y - y 3 )6(z -z 9 ,x- x 9 ) + (x - x 9 )6(y -y g ,z- z 9 ) = 0. 

This relation holds since 9 is alternating and is supported on We are done. □ 

Let us analyze in more detail the special case of hnite group G (this analysis is given in 
Dr2] and in a more expanded form in |RS| ). In this case, if k satishes the Jacobi identity, 
then by Proposition 12.81 k is supported on the set S. Therefore, the main commutation 
relation of the algebra H K takes the form 

[x, y] = t(x, y)+ ^2 9 g (x, y)g, 

geS,p(g)^l 

where r G (C[Kerp] (£> A 2 V*) G , and 9 g is a 2-form on V . It follows from the Jacobi identity 
that 6 g (x, y) = if x, y G Ker(l — g\v)- Since g is ^-invariant, this implies that the kernel 
of 9g contains Ker(l — g\v), and hence 9 g is uniquely determined by g up to scaling. We 
also see that k(x, y) is given by formula (|2.12|) for an appropriate 9. 

Furthermore, to secure the G-invariance of the right hand side of the commutation 
relation, we need to require that the family of forms 9 g is G-invariant. If C C S is a 
conjugacy class, then a nonzero invariant family {9 g ,g G C} is unique up to scaling if 
exists, and it exists if and only if the rank of 1 — g\y is exactly 2, and the centralizer Z g 
of g G C acts trivially on the 1-dimensional space A 2 Im(l — g). Let us call such conjugacy 
class admissible, and denote the set of admissible conjugacy classes by 5 a( jm- Thus we 
have 

[x,y] = r(x,y) + ^ 6 g (x,y)g. 

Note that since g G Z g , we hnd that if g is admissible then det(g) = 1, so g is conjugate 
in GL(V) to diag(A, A _1 , 1, 1). This shows that the study of degenerate Hecke algebras 
H K essentially reduces to the case when G C SL(V). 

Remark 2.14. It is explained in |Dr2l IRSj that the class of algebras H K with PBW 
property for hnite G in particular contains degenerate affine Hecke algebras for finite 
Coxeter groups, introduced by Drinfeld and Lusztig. 
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2.4 Continuous Hecke algebras and Symmetrie pairs. Let (G, K) be a Symmetrie 
pair of complex reduetive groups. Let g, 6 be the Lie algebras of G, K. Then we have the 
Standard decomposition g = 6 p. Let A(G, K) be the algebra of algebraic distributions 
on G set-theoretically supported on K. It is generated by ö(K)* and p, with appropriate 
relations. 

It is clear that A(G, K) is isomorphic to H K (K) for a particular distribution k, given by 
n(x,y) = [x,y] 6 t C Ö(K)*, where x,y G p (here we regard elements of t as distributions 
on K supported in the first order neighborhood of the identity). This means that A(G, K) 
is a continuous Hecke alegbra. 

In fact, one also has the converse result. 

Theorem 2.15. Assume that K C GL(E), and k : A 2 E f C Ö(K)* is an invariant 
map, such that Kern = 0, and the algebra H K (K) satisfies the PBW property. Then H K (K) 
comes from a Symmetrie pair, as above. Moreover, the group G is semisimple. 

The proof of the theorem is based on the following lemma. 4 

Let K be a reduetive algebraic group over C with Lie algebra t, and E a faithful 
algebraic representation of K. Let k : /\ 2 E — > t be a homomorphism of K-modules, such 
that KerK = 0. Let g = 6 © E. We have a natural linear map [, ] : A 2 g — > g, which 
extends the Lie bracket on 6 and the natural 6-actions on E, and satisfies the condition: 

Lemma 2.16. Suppose that [,] is a Lie bracket on g. Then g is a semisimple Lie algebra. 

Proof. Let B s , denote the Killing forms of the corresponding Lie algebras. We have 
B s (a,b) = Bz(a,b) + Trß{ab) for a, b £ t. Since E is faithful, for any o / in the Lie 
algebra 6r of a compact form Kr of K, we have B s (a, a) < 0. Hence the kernel of B g is 
contained in E. This kernel is an ideal in g, so it is contained in the kernel of k. Hence it 
must be zero, so B g is nondegenerate and g is semisimple. □ 

Proof. (of Theorem I2.15J) . Let g = t(BE. Since H K satisfies PBW property, we have 
g C H K (K). Then g is closed under bracket, so it is a Lie algebra as in Lemma 12.161 It 
follows from Lemma 12.161 that g is semisimple. We are done. □ 

3 Continuous symplectic reflection algebras and Cherednik 
algebras 

In this section we will consider the special case when the representation V has a G-invariant 
symplectic form u. 

4 The first author is grateful to D. Vogan for expiaining to him a proof of this lemma. 
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3.1 Continuous symplectic reflection algebras. Let S 3 $be the closed subscheme 
of G defined by the equation poA 3 (l- g\y) = 0, where p : A 3 V — > V is the projection 
defined by contracting the first two components using u. 



Theorem 3.1. Let t G (ö(Kerp)*) G and c G C(S). If 



k(x, y) = u(x, y)t + w((l - g)x, (1 - g)y)c for all x, y G V, 



(3.2) 



then the PBW property holds for H K . 

Proof. We shall show that (|2.5|) holds when n is of the form (|3.2|) . First note that {v—v 9 )t = 
0. Hence, we have to show that f(x,y,z;g)c = 0, where 

f{x,y,z;g) := (z- z 9 )uj(x-x 9 , y-y 9 ) + (x - x g )u(y -y g , z- z 9 ) + (y-y 9 )u)(z -z g , x-x 9 ). 



Let S be the set of symplectic reflections in G, i.e. elements s G G such that the rank 
of the Operator 1 — s on V is < 2. It is easy to see that the set of closed points of £ is 
SUQ, where Q is the subvariety of elements g G G such that (1 — g) 2 = 0. In particular, 
any semisimple element g G S(C) belongs to 5, and if A ^ 1 is an eigenvalue of g, then 
u((l-g)x, (l-g)y) = (1 - A)(l - A" 1 )w 9 (x, y), where u g (x,y) = uj(x',y'), x',y' being the 
projections of x, y on the image of 1 — g along the kernel of 1 — g. Hence, if G is a finite 
group and V is a faithful representation then the algebra of Theorem 13.11 is exactly the 
symplectic reflection algebra of jEG . 

Thus, algebras H K with k being of the form (|3.2|) are natural continuous analogs of 
symplectic reflection algebras of |EG| . We will therefore call H K a continuous symplectic 
reflection algebra. 

3.2 Continuous Cherednik algebras. Consider separately the special case V = rj©f)*, 
G C GL{\)) C Sp(V), where f) is a complex vector space, and the form u is the pairing 
between and f)*. In this case, let ^ be the closed subscheme of G defined by the equation 
A 2 (l — <?!(,) = 0. Obviously, C and the set of closed points of ^ is the set S of all 
complex (=symplectic) reflections in G, i.e., s G G such that the rank of 1 — s\h is < 1. 

Theorem 3.3. Let t G (C(Kerp)*) G and c G C(^). If n(x,y) = for x, y G fj orx,y£t)*, 
and one has 



This is clear, since c is supported on E. 



□ 



k{x, y) = {y, x)t + (y, (1 - g)x)c 



for all x G h*, y G f) 



(3.4) 



£/ien i/ie PBW property holds for H K . 
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Proof. We shall show that (|2.5[) holds when k is of the form (|3,4j) . Similarly to the proof 
of Theorem 13. 11 we have to show that hi(x, y, z; g)c = 0, fi2(x, y, u; g)c = 0, where 

hi(x,y,z;g) := (z - z 9 )(x,y - y 9 ) - (x - x 9 )(z,y - y 9 ), 

h 2 (x,y,u;g) := (u - u 9 )(x,y - y 9 ) - (y - y 9 )(x,u- u 9 ), 

where x,z G f)*, y,u & t). It is clear that h(x,y, z; g) is antisymmetric in x,z, while 
fi2(x,y,u; g) is antisymmetric in y,u. Hence, for any x,y,z,u the functions h\(x,y, z; g) 
and ti2(x,y,u;g) belong to the ideal of the scheme This implies that h%c = 0, h^c = 0, 
as desired. □ 

Note that if g £ ^(C) is not unipotent, and A ^ 1 is the nontrivial eigenvalue of g 
in ()* then (y, (1 — g)x) = (1 — X)(y',x'), where x',y' are the projections of x,y on the 
image of 1 — g along the kernel of 1 — g. Hence, if G is a finite group and f) is a faithful 
representation then the algebra of Theorem 13,31 is exactly the rational Cherednik algebra 
of [Eüj . 

Thus, algebras H K with k being of the form (|3.4j) are natural continuous analogs of 
rational Cherednik algebras. We will therefore call H K a continuous Cherednik algebra. 

Theorem 3.5. Assume that () is an irreducible faithful representation of G of real or 
complex type (i.e., it has no nonzero invariant skew- Symmetrie form) . 7/H K is a continuous 
Hecke algebra, then it is a continuous Cherednik algebra, i.e. k has the form given in 
Theoremün\ 

Proof. For the proof we will need the following proposition. 

Proposition 3.6. If the PBW property holds for H K , then K>(x,y) = whenever x,y are 
both in\), or both int)*. 

Proof. Suppose x,y G t). It follows from (|2,9[) that (z\x,y) = for all z S rj*. Hence, it 
remains to show that k(x, y) annihilates the constant functions on G. This follows from 
the identity (f\ 2 f)*) G = 0. Similarly for x,y € f)*. □ 

Now let us prove Theorem 13.51 Let z,x € f)*,y € f). By the Jacobi identity ()2.9j) and 
Proposition I3.fi[ we have 

(z\y,x) = (x\y,z). 
Taking the inner product of both sides with u € h, we get 

(u, (1 - g)z)n(y, x) = (u, (1 - g)x)n{y, z). (3.7) 

On the other hand, we have 

(n-n 9_1 ) K (y,z) = (y - y^Mu, z); (3.8) 
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this follows from the Jacobi identity by applying g . Let us now move 1 — g to the left 
in the right hand side of Q3.7JI . and apply (|3.8j) . This gives 



(«, (1 - 9)z)n{y, x) = (y, (1 - g)x)n(u, z). 



(3.9) 



Let ei be a basis of f), e* 
Then (|3,9() implies 



the dual basis of f)*. Let t^- = (e^, (1 — g)e*),Cij 





(3.10) 



Let us regard all our distributions as distributions on GL(Jj). We claim that Cjj = 
for some distribution b on GL(fj). Indeed, by (|3.1Ü|) . {%■} is a 1-cocycle in the Koszul 
complex of the C?(GL(f)))-module ö(GL(t)))* for the sequence of elements {%}. But this 
complex is exact, since ö(GL(i)))* is dual to a free module, and the sequence {Uj} is 
regulär. Thus, {%} is a coboundary of some element 6, as desired. 

Thus we have 



The distribution b is defined up to adding a multiple of 6\, so it is invariant under G. 

Since k is supported on G, we find that (y, (1 — is supported on G for all x,y. 

Thus, for any function (j) vanishing on G, b<p = j((j))5i, where 7 is a scalar-valued G- 
invariant linear functional such that j((f>f) = j((p)f(l), f S ö(GL(t))). This means that 7 
has the form 7(0) =< (d(ß)(l), Y >, where Y G fll(fl) is an invariant element (i.e., Y = t-Id 
is a scalar Operator, since f) is irreducible). Let c = b — Y . Then we get 



and c is supported on G. 

It remains to show that c is (scheme-theoretically) supported on ^f. But this is clear 
from the Jacobi identity: 



Corollary 3.11. If f) is a faithful irreducible representation of real or complex type, then 
any continuous symplectic reflection algebra is a continuous Cherednik algebra, and vice 
versa. 

Proof. It follows from Theorems 13. 11 15*31 that any continuous symplectic reflection algebra 
is a continuous Cherednik algebra. To prove the converse, let e% be a basis of f) and e* the 
dual basis of f)*. Then on ^ we have 



u(0--g)x, (i-9)y) = ^(i 1 -g) x , e i)( e h i 1 -g)y) = - ^2(g%, i 1 -g)ei)(e*, (i-g)y) = 



K(x,y) = (y, (1 - g)x)b 



k(x, y) = t(y, x)8i + (y, (1 - g)x)c, 



c{y, (1 - g)x)(l - g)z = c{y, (1 - g)z){\ - g)x. 



The theorem is proved. 



□ 



1 
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-^2(9X, (l - g)y)(e* , (l - g)ei) = - g)x, y)(e* , (1 - gfe) = <f>(g){y, (1 - g)x), 

i i 

where <fi(g) = Tr(l — g)\^. Now let c be a G-invariant distribution on *S>, and c be a G- 
invariant distribution on G such that (f)c = c (it exists since <j) is not a zero divisor in Ö(G) , 
as for any element g / 1 of a maximal compact subgroup of G, one has </>(g) > 0), so the 
multiplication Operator by 4> is injective on Ö(G) G and hence surjective on (Ö(G) G )*). 
Then we have 

^((1 - g)a;, (1 - #)y)c = (y, (1 - ff)x)c. 

From the Jacobi identity it is clear that c is supported on £, which implies the required 
statement. □ 

Corollary 3.12. Let (G,K) be o Hermitian Symmetrie pair (with semisimple G). Then 
A(G, K) is a continuous Cherednik algebra. 

Proof. Let g, t be the Lie algebras of G, K, Then g = t © p + © p_, and we have a linear 
representation K — > GL(p + ) Also, we have [p+,p + ] = [p_,p_] = 0, while [p + ,p_] C t. It 
follows (after a simple computation) that A(G,K.) = H K (K), where k is such that t = 
and c = A6\, A being the Laplacian on K attached to the Killing form of G restricted to 
K. □ 

Remark. We see that it is possible that the scheme Vt/G is not reduced (i.e., $/G / 
S/G), and hence C(S) 7^ C(^). Indeed, for Hermitian Symmetrie pairs one often has 
S = {1}, and yet one has a distribution n(x,y) which satisfies the Jacobi identity and is 
not a multiple of ö\. 

3.3 Examples. 

3.3.1 The case G = GL n . 

Let f) be an n dimensional vector space, and let G = GL n be the general linear group of fj. 
In this case, we have an isomorphism S/G — > C* : s 1— > A = Tr|(,(s) — n + 1. Thus, C(S) is 
the space of Fourier series J2m& c m^ m - Also, the scheme ^ is well known to be reduced, 
so we have * = S. Thus in this case C(*) = C(S). 

3.3.2 The case G = O n . 

Let 1) be an n dimensional vector space with a nondegenerate Symmetrie bilinear form, 
and let G = O n be the orthogonal group of fj. Note that for n = 2 this example can be 
regarded as a continuous limit of symplectic reflection algebras of finite dihedral groups. 
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In this case S consists of 1 and the conjugacy class of orthogonal reflections, so S/G 
consists of two points and hence the space C(S) is two dimensional. However, it turns out 
that ^ is not reduced in this case. Luckily, we have the following proposition. 

Proposition 3.13. For G = O n , C(^) = C(5). 

Proof. First we claim that \P is reduced near every orthogonal reflection. Indeed, let s = 
diag(— 1, 1, 1) G S. Then the tangent space T S S is the space [s, q] of Symmetrie matrices 
A = iflij) such that a,ij = unless i ^ j and i = 1 or j = 1. Now assume that s + HA is 
a point of Vi/ over the ring of dual numbers C[h]/h 2 . The nontrivial minors of 1 — s — HA 
are Au,lj = — so we get ety = if i, j > 1. We also have det(s + HA) = — 1 + hau, 
so an = 0. Hence, A £ X^S 1 , and thus T,^ = T S S. 

It remains to consider the component of the identity in ^. Writing g = e A , where 
A is skew-symmetric, we may (G-equivariantly) identify with the subscheme of so n 
defined by the equation A 2 A = 0. 

We claim that the ring ö(^\) is spanned by funetions 1 and a^, i < j, with triv- 
ial multiplication. Indeed, it is easy to see that l,a« are linearly independent, and 
dijaki = aijaki — aaakj = 0. Also, a^au is Symmetrie under the transposition of j, l 
but antisymmetric under the transposition of i,j and k, l, so it is zero. 

Thus, 0(ß?i) = C A 2 f) as a G-module, and hence Ö(ß?i/G) = C, as desired. □ 

We conclude that for G = O n the continuous Cherednik algebras form a 2-parameter 
family (out of which only one is essential, because of scaling). 

3.3.3 The case G = Sp 2n - 

Let V be a 2n dimensional vector space with a nondegenerate skew-symmetric bilinear 
form, and let G = Sp2 n be the symplectic group of V. We have an isomorphism S/G — > 
C : s i—> w = X + A" 1 = Tr|v(s) — 2n + 2. Thus, C(S) is the space of Fourier series 
J2 m ez c m.A m with c m = c_ m . 

Theorem 3.14. We have: (i) § = S, and S is an irreducible affine algebraic variety; 

(ii) if the PBW property holds for H K , then n(x,y) is of the form h'J. with c € 
C($) = C(5). 

Proof. (i) We first show that $ is reduced (i.e., coincides with S). Let ScS be the subset 
of elements which are not unipotent. Let Gr{2n — 2, In) be the Grassmannian variety of 
2n — 2 dimensional subspaces in V. The map S — > Gr{2n — 2, 2n) : g \— * Ker(l — g) is a 
smooth submersion, with open image (the set of nondegenerate codimension 2 subspaces 
of the symplectic space V). Hence S is smooth. 
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Any conjugacy class in S contains a representative of the form 

(ab \ 

c d 



g 



1 



V 



We claim that the tangent spaces T g <& and T g S are equal. Indeed, let g + hQ (Q = (qij)) 
be a point of $ over the ring C[h]/h 2 . Then we have Tr{g~ l Q) = 0, and the equations 
A 3 (<7 + HQ — 1) = imply that = for all i,j > 3. On the other hand, T g S is spanned 
by matrices of the form [g, M] where M € $p(2n), and matrices Q = gP, where P has the 
form 

faß \ 

7 ö 

" where a + 5 = 0. 



V o ) 

Therefore, T 9 $ = T g S, and so is reduced at the formal neighborhood of each g £ S. 

The set of points of S where $ is not reduced is G-invariant. Therefore, to prove 
that this set is empty, it remains to show that $ is reduced at the formal neighborhood 
of 1. Let A be the algebra of functions on the formal neighborhood of 1 in the scheme 
A 3 (l — g) = 0, where g € G. Let Aq be the algebra of functions on the formal neighborhood 
of in the scheme A 3 M = 0, where M G sp(2n). Explicitly, A = C[[l - g]]/(A 3 (l - g)) 
and Aq = C[[M]]/(A 3 M), where g = (gij) and M = (rriij) are matrices of formal variables. 
The pullback of the exponential map gives an isomorphism exp* : C[[l — g]] — > C[[M]]: 
g^ l + M + M 2 /2! + --- . Nowexp* : A 3 (c/-1) h-> A 3 (M(1 + M/2! + M 2 /3\ + • • • )). The 
element 1 + M/21 + M 2 /3! + • • • is invertible, and hence exp* gives an isomorphism from 
A to Aq. By |Ku| Theorem 1], Aq is reduced, so A is also reduced. 

The irreducibility of $ = S follows from the easy fact from linear algebra that S (which 
is irreducible) is a dense subset of S. 

(ii) Let S' be the subset of elements of S which are not unipotent and do not have 
eigenvalue —1. By part (i), S is irreducible, and S' is an affine open set in S, defined by 
the condition f 2 - 4 / 0, where f(s) = Tr(s) - 2n + 2. Hence O(S) C O(S'). Thus, we 
have a surjective G-invariant map £ : O(S')* — > Ö(S)* . 

The group G acts on S', with S' /G = C \ {2, —2}, and the stabilizer of each point is 
isomorphic to K = Sp(2n — 2) x C*. Thus, S' is a fibration over C \ {2, —2} whose fiber 
is G/K, and the action of G is fiberwise. 

Let k be a distribution satisfying the PBW property. Then, as follows from Proposition 
EHland part (i), k € (O(S)* A 2 V*) G . Let K be a lifting of k in (0(5"')* ® A 2 V*) G . Since 
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S' is fibered over C\{2, —2} with über G/K, we may view 7c as a distribution on C\{2, —2} 
with values in the space (A 2 V*) K , which for n > 1 is a two dimensional space spanned by 
co(x,y) and w((l — y)x, (1 — g)y) (where g is a semisimple element in S whose centralizer 
in G is K). In other words, we have 

k(sOO, y) = a{g)u(x, y) + ß{g)u((l - g)x, (1 - g)y), 

where a,ß are G- invariant distributions on S' . The Jacobi identity for k implies that 

(z — z a )~K(x, y) + (x — x 9 )7c(y, z) + (y — y 9 )7c(z, x) = 

modulo distributions that vanish on O(S). This imposes no condition on ß, and the 
condition on a is that the distribution 

«(sXO* - z 9 )u){x, y) + (x - x 9 )u;(y, z) + (y - y g )u{z, x)) 

vanishes on O(S). This implies that a is scheme-theoretically supported at 1, so a = t8\. 
The result follows. 

If n = 1, then the proof is even simpler, since in this case (A 2 ^*)^ is a 1-dimensional 
space spanned by u>. □ 

4 Infinitesimal Hecke algebras 

4.1 The main theorem. Let be the Lie algebra of G. The universal enveloping 
algebra Ug is naturally isomorphic to the subalgebra of Ö(G)* consisting of all algebraic 
distributions set-theoretically supported at the identity element 1 of G, cf. |DG1 II, §6]. 
When the image of n : V x V — > Ö(G)* lies in Ug, let Jt? K = M? K {g) be the quotient of 
TV yj Ug by the relations (TOI) . 

We dehne a filtration on Jt? K by setting deg(x) = 1, deg(n) = for x G V, u G 
£^g. There is a natural map J^o = ^ ^0 — * 9 r which is a surjective graded 
algebra homomorphism, and we say that the PBW property holds for J4? K if this map is 
an isomorphism. Since H K (G) = 0(G)* ^HgJ^nig), the PBW property holds for J4? K (g) if 
and only if it holds for H K (G), i.e. when k satishes the Jacobi identity. Whenever this is 
the case, we call J4? K an infinitesimal Hecke algebra. 

Example 4.1. For instance, we can dehne inhniteimal Hecke algebras in the examples 
attached to Symmetrie pairs, see Section |2~H in this case it is easy to see that Jf K is the 
universal enveloping algebra of the Lie algebra of the group G. 

The aim of this section is to prove the following result when g is gi n or sp 2n . 

Theorem 4.2. The PBW property holds for J$? K (g) (i.e., it is an infinitesimal Hecke 
algebra) if and only if the relations \2.3\) are of the form \4-4\) when g = g{ n , and \4-3j) 
when g = sp 2n . 

The relations (|4,4[) and (|4.9j) will be given below. 
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4.1.1 The g = gl n case. 

Let f) := C™, V := © f)* and g := gl n = gl(Fj). We shall identify g* with g via the pairing 
g x g — > C : (A, B) i— > Tr (AB). We shall also identify Sq with Uq via the symmetrization 
map. For any x S [)*, 1/ G f), i G g, we may write 

(x, (1 - tA)'^) det(l - rA)- 1 = r (x, + n(s, y){A)r + r 2 (x, v){A)t 2 + ■■■ (4.3) 

where ri(x,y) G »Sg is a polynomial function on g. 

For each polynomial ß = ßo + ß\T + ß 2 r 2 + ■ • • G C[r], dehne the algebra = 
J4?ß(Ql n (C)) to be the quotient of T(fj © f)*) x Uq by the relations 

[x,x'}=0, [y,y']=0, [y,x] = ß r (x,y) + ßin(x,y) + ß 2 r 2 (x,y) -\ , (4.4) 

for all x, x' G rj* and y, y' G f). 

Remark 4.5. To expand the left hand side of Q4.3JI as a power series in r, we write 

(1 - tA)' 1 = 1 + tA + r 2 ^ 2 + • • • , 
det(l - r^)" 1 = e -l«»gdet(l-TA) = e -TrlQg(l-TA) = e E fc >o^Tr(^ fe /fc) 

= l + rTr(A) + r 2 (Tr(^) + ^) + .... 

Example 4.6. We have r (x,y) = (x,y). Thus, J^(g[ n (C)) = D(C n ) x Wgt n (C) (where 
D(C n ) is the algebra of polynomial differential Operators on C n ). 

Example 4.7. We have n(x,y) = y <8> x + (x,y)ld G gl n (C). Thus, J^(gl n (C)) = 
W(f) © rj* © gl n (C)) , where the Lie bracket on f) © f)* © gl n (C) is 

[(2/,^,^), G/^',^')] = - A'y,xA' - x'A, [A,A'} +r 1 (x',y) - n(x,y')). 
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There is a Lie algebra isomorphism 9? : h © f|* © fll n (C) — > sl n+ i(C) defined by 



V 



\ 



x 



/ 







A 



\ —x 



A 



V 



/ 







/ 





' n+l 




Aid ^ 


A 
n+l 







V 


-nA 
n+l 



if y € f), 

if x € f)*, 

if AesI n (C) c [ n (C), 
\ 

if A € C. 



Obviously, this example comes from the hermitian Symmetrie pair (SL n+ i,GL n ), as 
explained in Section ITH 

4.1.2 The g = sp 2n case. 

Let y = C 2n be a symplectic vector space with symplectic form cj, and g := sp(V). Again, 
we identify q* with g via the pairing gxg^C: (A, B) 1— > Tr(AB), and S*ß with Ug via 
the symmetrization map. For any x, y € V, A G 0, we may write 

a;(x,(l-r 2 yl 2 )- 1 y)det(l-r^)- 1 = 4( x ,y)(^)+£ 2 ( a ;,y)(A)r 2 +4(x,y)(^)r 4 + - • • (4.8) 

where ^(x, y) € Sq is a polynomial funetion on q. 

For each polynomial ß = ßo + ß2i~ 2 + ß^t 4 + • • • € C[r], dehne the algebra Jßß = 
J%ß(sp(V)) to be the quotient of TV x Uq by the relations 



[sc, y] = /3o40, y) + ß2^{x, y) + Ai&lOe, y) + 



(4.9) 



for all x, y € V. 

Remark 4.10. Note that A € sp(V) implies A k = (-l) k J T (A T ) k J, where J is the matrix 
of the form w in a symplectic basis. Thus, in (|4.8[) . cj(x, (1 — t 2 A 2 )~ 1 y) is skew-symmetric 
in x, y. We also have Tr(A k ) = if fc is odd, so the expansion of det(l — tA)" 1 contains 
only even power s of r. 
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Example 4.11. We have £o(x,y) = oj(x,y). Thus, J%{(sp(V)) = A n x Usp(V), where A n 
is the n-th Weyl algebra. 

Example 4.12. When n = 1, we have g = sl2- Let 



x c-ay' V 00 /' v 0-1 / V 10 

Note that A 2 = (a 2 + 6c)ld, so &2k(x,y) is equal to a constant times ui(x,y)A k , where 
A = ^- + e/ + /e is the Casimir element of Us\i- The algebra 3^ß{sX-i) was studied by 
Khare in IK1 



4.2 Proof of Theorem 14.21 Let G = GL n when q = gl n , and let G = Sp2 n when 
= 5p 2n . By Theorems EU EU EÜ3 and Corollarv EZEH the PBW property holds if 
and only if k is of the form ()3.2|) . Thus we have to find all c G C(S') such that n{x,y) is 
set-theoretically supported at 1 for all x,y G V, and then compute y) as an element 
in Uq. 

The q = Ql n case: By Proposition 13.61 we may assume that k(x, y) = if x, y are both 
in f) or both in fj*. Following Subsection lH.M.U we identify C(5) with the space of algebraic 
distributions on C*, so that c G C[A, j]*. Then n(x,y) is set-theoretically supported at 1 
for all x, y G V if and only if c = c(A) is a finite linear combination of S\ and its derivatives. 

Now let f) = C n be equipped with the Hermitian inner product (x, y) := x T y. We may 
identify f)* with t) using this inner product on fj. Let 

K:= U(n,C) = {geG,\g- 1 =g T }, 
s d :=diag(e iö ,l,...,l) G K, 8 G [0,2tt], 

S e := {w^lse^cif. 

For any s G Sg, denote by p s G the orthogonal projection of f) to the eigenspace of s 
with eigenvalue e* 9 . We have p s = v®v where v G f) is an eigenvector of s with eigenvalue 
e l8 and |v| = 1. We have to compute 

K(x,y) =t(x,y) + f c{6)( f (1 - e - ie )(x,p s y)s ds) dÖ 

=i(ar, y) + j (x, (v®v)y) ( (l - e - ie )c(9)e ie ^ v) dö) dt; 



=i(x,y)+ / (x,(v<&v)y)f{v®v)dv £ Wg, 
y|«|=i 

where x G f)*, y G f), c(ö) is a finite linear combination of 5q and its derivatives, and / is 
some polynomial depending on c. 
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Recall that we identify Uq with Sg via the symmetrization map. Let F m G Sq be the 
function on g defined by 

F m (A) := ( (Av,v) m+1 dv, A € g. 
y|«|=i 

dF m \ A (y®x) = (m + 1) / (x, (u<g)ü)y)(A;, w) m <fo. 

yi«i=i 

Thus k(x, y) corresponds, under symmetrization map, to a finite linear combination of 
functions dF m \A(y®x). 

We now compute F m (A). First, note that 



Note that 



/ (Av, v) 



m+l e -C(v,v) dy 



/ (Av,v) m+1 dvdr 

J \v\=r 

f°° r2m+2 n+i e -cr 2 d r-F m {A) 
Jo 



(m + n)\ 



C^^FrniA). 



On the other hand, we have 



-{((-A)v,v) dy 

=ir n det(C - Ay 1 . 

Hence, F m (A) is equal to a constant times the coemcient of r m+1 in the expansion of 
det(l — TÄ)~ l as a power series in r = £ _1 - Differentiating det(l — tÄ)" 1 and evaluating 
at B € g, we obtain 

9ß v ; det(l-rA) 

Setting 1? = y(g>x gives r(x, (1 — ri) -1 )/) det(l — rA) _1 , as desired. 

The g = sp 2n case: We let F = C 2n be equipped with the symplectic form lü(x, y) = 

\ 



v T Jy where 



/ 1 
-1 



J := 



1 
-1 
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Let 

K := U(n, H) = {g G GL 2n (C) | / J<? = J, <T X = g T }, 
s d :=diag(e iö ,e- iö ,l,...,l) G K, 8 G [0,2vr], 

(here EI is the division algebra of quaternions 5 . For any s £ Sj, we denote by p s the 
w-orthogonal projection of V to Im(l — s). 

Notice that if s G Sg, q G V, and = e t9 q, then sJg = J~sq = e~ %e J~q. Take any 
q G V with g T g = 1. For any y G V, we have y = u>(Jq, y)q + u>(y, q)Jq + y', where y' is 
u;-orthogonal to q and Jq. Let 

s 9 (0) := e iep9 , where p q := qq T + Jqq T J G iLieif. 

Note that if q = (1, 0, . . . , 0) T , then p g = diag(l, —1, 0, . . . , 0) and s q (9) = se- 
het x,y G V. Now 

u(x,p Sq ( 9) y) =u(x,Lj(Jq,y)q + uj(y,q)Jq) 

=w(x, q)uj(Jq, y) - to(y, q)u(Jq, x) 
=q T (yx T - xy T )Jq 
=uj((xy T - yx T )q,q). 

According to Subsection 13.3.31 we have to compute 

K(x,y) =tuj(x,y) + f (l - e i6 )(l - e- i6 )c(0)( I uj(x,p s y)s ds) d6 

J J Sß 

=tüü{x, y) + J oj({xy T - yx T )q, q) ( (1 - e i6 ){l - e- ie )c{6)e iep " dö) dq 

=tu(x, y)+ u({xy T - yx T )q, q)f{p q ) dq G Kg, 

J\q\=l 

where c{9) is a finite linear combination of 5q and its derivatives (of even order), and / is 
some polynomial. 

Recall that we identify Uq with Sq via the symmetrization map. Let F' m G Sq be the 
function on q defined by 

F' m {Ä) := / u{{xy T - yx T )q, q)Tr(Ap q ) m dq, A G Q. 

J\q\ = l 

5 Quaternions appear because U(n,M) is the maximal compact subgroup of Sp(2n, C) 
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Now 

Tr(Ap q ) = Jr(Aqq T ) + Tr(AJqq T J) = q T Aq + q T J AJq 
= q T Aq + q T JA T Jq = 2q T Aq = -2u(AJq, q). 

Thus, 



Fl(A)= f u>(Bq,q)u>(Cq,q) m dq 



l\ q \=l 

where B = xy T - yx T and C = -2AJ. Dehne F m e S(gl 2n (C)) by 



F m (M) := / uj(Mq, q) m+l dq, M G gl 2n (C). 
J\q\=i 



'\q\ 

Note that dF m \ c {B) = (m + l)i^(A). 

We compute F m as in the previous case, by considering 



[ w(Mq,q) 
Jv 



m+l p -(q T q 



Now, 



E 7 — ZTTT / "(Mq,qr +1 e-^ T «dq= [ 



e - riC - M Tj )qdq 

v 



=7r 2n det(C -M T J)- L 
=tt 2h det(C + JM)" 1 

Hence, F m (M) is equal to a constant times the coefhcient of r m+1 in the expansion of 
det(l + t,JM)~ 1 as a power series in r. Differentiating det(l + tJM)~ 1 and evaluate at 
B, we get 

-Jt{tJB{1 + rJM)" 1 ) det(l + rJM)" 1 . 
Substituiting ß = xy T — yx T , M = C = —2AJ, and replacing r by r/2, we get 

-TJr(j(xy T - yx T )(l - tA t )~ 1 ) det(l - r^)" 1 / 2 
=r(w(x, (1 - TA)~ l y) - tü(y, (1 - tA^x)) det(l - tA)- 1 /2 
=tu(x, (1 - t 2 ^ 2 )- 1 ^) det(l - tÄ)- 1 . 

□ 
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5 Representation theory of continuous Cherednik algebras. 



In this section we will consider the basics of representation theory of continuous Cherednik 
algebras H K (G), where G C GL(f)), V = f) © f)*. For simplicity we will assume that f) is a 
faithful representation (so t € C). 

By a representation of H K (G) we always mean a continuous representation, i.e., a 
locally finite representation of G together with a compatible action of V satisfying the 
main commutation relation. 



5.1 Dunkl Operators. Let g G GL(fj) be an element such that the Operator 1 — g 
has rank 1. Let / G C[h]. Then the function f(u) — f(gu) is divisible by ß(u), where 
ß G Im(l — <7 -1 *) is a basis element. Thus for any y G f) we can dehne the regulär function 

K f (g,y,u) :=ß{y) 

on {Sgl($) \ {1}) x f) x f) (it is independent on the choice of ß). Since S GL ^ is normal, 
this function extends to g = 1 (by zero). Hence Kf may be regarded as a regulär function 
on the scheme ^x[)x[)(asf = ^GL(t)) n G). 

Now let c be a G-invariant distribution on ^. For any y G f), dehne the Dunkl Operator 
_D y on C[h] by the formula 

W)(«)=W + (^/( ? .y-«))' 

(Here the last term means the evaluation of the distribution c on the function Kf(?,y,u) 
of g for hxed y,u). 

Theorem 5.1. Let n(x,y) = —t(x,y) — c(x, (l — g)y), and H K be the corresponding contin- 
uous Cherednik algebra. The assignment x — > x for x G fj*, # — > g (action of g on C[t)]), 
and y — > defines a representation of H K on C[f)]. 

Proof. Let H+ be the subalgebra of H K generated by f) and Ö(G)*. Let C be the trivial 
representation of this algebra, where y G f) acts by zero, and # G G by 1. Let M := 
H K <8> H + C. It follows from the PBW theorem for H K that M = C[f)], where x G f)* acts by 
x and g £ G by g. It remains to compute the action of y. Let xi, ...,x r G fj*. Then we 
have 

y o xi...x r = [y,x 1 ...x r ] ■ 1 = ^ Xi...Xj-i[y, Xj]x i+1 ...x r • 1 = 

i 

y^(*(y, Xj)xi...Xi_ix i+ i...x r • 1 + (c, (y, Xj - xf )xi...Xj_ixf +1 ...x^ • 1) = 

i 

iö y (xi...x r ) + (c,^...^?,^)). 

(Here as usual (c, /), for / G 0(G), is the evaluation of the distribution c on the function 
/). The theorem is proved. □ 
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The representation defined above is called the Dunkl-Cherednik representation of the 
Cherednik algebra H K . If G is finite, it is the usual Dunkl-Cherednik representation of the 
rational Cherednik algebra. 

Corollary 5.2. We have [D y ,D y i] = for any y,y' € f). 

5.2 Category ö. Consider a continuous Cherednik algebra H re , with t = 1 and arbitrary 
c. Let dim f) = d. Like for usual Cherednik algebras, we can dehne the Euler element 



where is a basis of f) and x% the dual basis of fj* . This element commutes with Ö(G)* . 
Proposition 5.3. We have [h, x] = x, [h, y] = —y for x £ h*, y G h. 
Proof. We have 



Definition 5.4. Category ö for H K consists of finitely generated h\ K -modules which are 
locally finite with respect to h, with real part of the spectrum of h bounded below. 

The properties of category ö are as usual; let us summarize some of them. The proofs 
are Standard, see GGÖR]. 

1. Any object of ö is a finitely generated C[f)]-module. 

2. For every irreducible finite dimensional G-module Y we can define the Verma 
module M(Y) = H re (g> H + Y (where y € f) act in Y by zero). In particular, M(C) is the 
Dunkl-Cherednik representation. The module M(Y) has an irreducible quotient L(Y), 
and every irreducible object in ö is isomorphic to L(Y). 

Remarks. 1. In the case of Hermitian Symmetrie pairs (H K = A(G, K)), the modules 
L(Y) (for appropriate 1") are the representations of "holomorphic discrete series". 

2. For c = 0, one has M(Y) = L(Y), and category ö is semisimple. This means that 
this is so for Weil generic c, i.e. outside a countable union of algebraic hypersurfaces. (An 
interesting problem is to find exactly for which c this happens.) 

3. For any module M E ö, we can define the character \M = TrM(gt h ), which is a 
series in t with coefficients in the representation ring of G. For example, 




[h, x] = [c, x) + ^2 x AVii x ] = [c,x]+x + (1 - g) 



X ■ C = X. 



The second identity is proved similarly. 



□ 



Xm(y) 



det |[,*(1- gt) 
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where cy is the eigenvalue of c in Y. One of the main problems of representation theory 
of H K is to find the characters of L(Y). 

To keep this paper short, we will not develop representation theory of continuous 
Cherednik algebras systematically. Rather, to give the reader the flavor of this theory, 
we will study the question of reducibility of the Dunkl-Cherednik representation for G = 
GL(fj) and G = 0(fj). 6 

5.3 Representations of H K for G = GL(§). 

Proposition 5.5. (i) The Dunkl-Cherednik representation M(C) has singular vectors in 
degree N if and only if 

c s jv r - cc = N. (5.6) 

Thus M(C) is irreducible iff equality 15. ö]) fails for all N > 1. 

(ii) If N is the smallest positive Solution of 15. b]) then L(C) = ®j^ 1 iS ,J 'fj* is the quotient 
o/C[h] by the ideal generated by polynomials of degree N. 

Proof. (i) The action of y E f) defines a homomorphism of G-modules f)®S^fy* — > S 1 ^ -1 !)*. 
There is only one such homomorphism up to scaling, so its vanishing is equivalent to one 
linear equation in c. If this equation is satisfied then there is a nontrivial homomorphism 
M(S N t)*) — > M(C) of degree N, which implies l|5.fij) . Thus the linear equation in question 
is exactly (|5.ö|) . as desired. 

(ii) is clear from the proof of (i). □ 

5.4 Representations of H K for G = O(f)). Let G = 0(t)), dim(fj) = d. We fix an 
isomorphism t) — > f)* defined by the inner product on f). Let s € 0(1)) be an orthogonal 
reflection, and Ad(G)s = S \ {1} be the conjugacy class of renections. 

In this case, it is sufficient to consider c = kA, where A is the integral over the 
conjugacy class of renections (=invariant projection to the trivial G-subrepresentation 
ö(Ad(G)s) —>■ C), and k € C. Thus we will denote the algebra H re with t = 1 by H^. 

The algebra has an sl(2) subalgebra, given by F = ^^2xj, E = —^^2yf, H = 
— \{ x iUi + UiXi) (it is easy to show that H = — h). 

The highest eigenvalue of H on the Verma module M(Y) is 

w 2 dim(Y) 

This shows that in order to have a nontrivial map M(Y) — > M(Y'), the number k must be 
rational. Thus, if k is not rational, the category ö = is semisimple, as MiY) = L(Y) 
for all Y. 



6 Note that for G = GLd, the representation theory of H K is in some sense a "deformation" of represen- 
tation theory of the Lie group SU(d, 1) (with infinitely many parameters) 
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Definition. The Shapovalov form M(Y) x M(Y*) — > C is the bilinear G-invariant 
form coinciding with the Standard one on Y x Y* , and satisfying the equations (yv,w) = 
(v,y*w), (v,yw) = (y*v,w), where y € h, and y* S f)* is the element corresponding to y 
under the identification fj — > rj*. 

It is easy to show that the Shapovalov form exists and is unique. Let K (Y) be the 
kernel of the Shapovalov form, then L(Y) = M(Y)/K(Y). 

Let us study the kernel K(Y). Any representation of is in particular a representation 
of the dual pair (G,sl(2)). Moreover, for generic k, M(Y) = ® n >üQ{n,Y) (g> M\ Y n*_ n , 
where M„ is the highest weight Verma module over sl{2) of highest weight fj,, and Q(n, Y) 
is the finite dimensional space of highest weight vectors of degree n under sl(2). From 
character considerations, it is easy to see that the representation Q(n, Y) of G is isomorphic 
to Harm(n) (g) Y, where Harm(n) is the space of harmonic polynomials of degree n. 

Let us now study the structure of the Dunkl-Cherednik representation M(C); i.e., 
Y = C, Xy(k) =: X(k) = —d/2 — k. Let L m denote the finite dimensional irreducible 
representation of sl(2) with highest weight m (m € Z+). 

Proposition 5.7. If k = —d/2 — m, m E Z +) then L(C) = (B™ =0 L m _ n (g) Harm(n). 
Otherwise L(C) = M(C). 

Remark. Let d = 2, and k = —1 — m. Then the representation L(C) has dimension 
k 2 and is a limit of representations of dimension k 2 for finite dihedral groups G found in 
Theorem 3.3.1 of |Chj (first row of the table). 

Proof. Since all representations Harm(n) are irreducible and non-isomorphic, for any k 
we have M(C) = © n >oM(n) <8>Harm(n), where M[n) is an s£(2)-module having the same 
character as M\^_ n . But since F = r 2 := Yl x h ^ e m odule M(C) has no singular 
vectors under F. Hence M(n) = M A ( fe )_ n , and thus M(C) = ® n >oM\(k)-n ® Harm(n). 

Thus the Shapovalov form on M(C) is the sum of the tensor product forms on 
M\(k)-n ® Harrain) with coefficients. For nonnegative integer X(k) — n, the Shapo- 
valov form must vanish on M_x(k)+n—2 C Muty_ n . Furthermore, we claim that all 
M\(k)-n ® Harm(n) with n > \(k) belong to the kernel K{Y). Indeed, K(Y) is a G- 
invariant proper graded ideal in C[h]. So its zero set is either the cone r 2 = or the origin. 
In particular, r 2n \ L ^Y) = for some n. This proves the claim. 

Thus, the zero set of K(Y) is the origin, and L(Y) is identified with a SL(2) x O(d)- 
subrepresentation of the finite dimensional representation ©^=o-^A(fc)-n ® Harm(n). 

It remains to show that this subrepresentation is in fact the entire representation. To 
do so, note that the component Lxua corresponding to n = must occur, as it contains the 
highest weight vector. This implies that r 2p ^ in L(C) for p = 0, X(k). Now consider 
the component corresponding to some n < X(k). Consider its s/(2)-highest weight part, 
Harm(n). This is a representation of 0(d) defined over the real numbers, so it has a real 
basis fi orthonormal under the invariant inner product. Consider the function F = ff- 
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This function is rotationally invariant and generically positive on the real subspace (as 
it is a sum of Squares), so it is a nonzero multiple of r 2n . This implies that the vector 

fi ' fii where the first factor /j belongs to and the second one to L(C), is nonzero. 
So at least one fi € L(C) is nonzero, and the entire n-th summand must occur in L(C). 

The proposition is proved. □ 

Remark. In fact, the proof of the proposition shows that any representation of 
from the category ö which does not have füll support in f) is finite dimensional (as it 
is s£(2)-finite). So, the quotient of category of ö by the category of finite dimensional 
representations maps injectively into RepO(n — 1) through the fiber functor at a generic 
point of f). This functor is somewhat analogous to the KZ functor from jGGÜRj . 



6 Case of wreath-products 

6.1 Wreath products. Many interesting examples of continuous symplectic reflection 
algebras may be obtained by means of wreath-product type construction. Specifically, 
let r be a reductive subgroup of SL2, and G = S n x T n C Sp2 n - When T is finite 
(i.e., corresponds to a simply-laced affine Dynkin diagram), the corresponding symplectic 
reflection algebras were considered in |EG| . Thus here we will consider only the cases 
when r is infinite. There are three such subgroups T: 

Example 6.1. r = C* = GL\ (the maximal torus). In this case S/G = C* U {s}, where 
s the conjugacy class of Sij. 7 

Example 6.2. T = O2 (the normalizer of the maximal torus). In this case, S/G = 
C* U {s, er}, where a is a non-diagonal element of 

Example 6.3. T = SL 2 . In this case, S/G = C* U {s}. 

These correspond to the infinite affine Dynkin diagrams Aqq, -Doo> ^00/2 depicted in 
Figure 1 by the Standard rule of McKay's correspondence (see below). 

Proposition 6.4. Let G be as in examvle \6.1lö. 6 A or \6.'A Then the most general contin- 
uous symplectic reflection algebra for G is obtained by taking c G C(5"). 

Proof. In Example 16.11 it is easy to show that ^ is reduced (i.e., \P = S), and thus the 
statement follows from Corollarv 13.111 

In examoles 16.216.31 we have E = So U Ad(G)s,S = So U Ad(G)s, where So and So 
are the intersections of £ and S with Go := T n . Also, one has C[Go] G ° = C[T\, T n ], 
where the funetions Tj are defined by the formula Ti(7i, 7„) = Tr(7j — 1). The defining 
equation of S introduces the relations T{Tj = 0, % 7^ j. This implies that S/G = S/G, and 
the statement follows. □ 

7 In all three examples, the class s is absent if n — 1. 



26 



Aoo -Doo ^-oo/2 

Figure 1: Graphs associated to GLi, O2, and SL2. 

We see that in all three examples, for n > 1 one has C(S) = C(Sq) © CA, where A is 
the integral over Ad(G)s. Moreover C(So) is naturally isomorphic to C(r) (as r is S for 
n = 1); we will identify these Spaces using this isomorphism. 

6.2 Representations of wreath product algebras. The theory of wreath product 
algebras for infinite groups T is rather similar to the theory in the finite case. As an 
example let us work out finite dimensional representatations of H K which are irreducible 
as G-modules. For finite groups T, this was done in [M]. Since the proofs in the infinite 
case are the same as in the finite one, we will omit them. 

As before, we will denote by T one of the groups GL\, O2 or SL2, and let G := S n K T n . 
We shall write L for the 2-dimensional tautological representation of T; V = L® n . 

Let {Y{}i e j be the set of finite dimensional irreducible representations (up to isomor- 
phism) of r. We associate a graph to T in the following way: the set of vertices of the 
graph is /, and the number of edges joining i,j El is the multiplicity of Yi in L®Yj. See 
Figure 1, where ^4oo, Ax,, and A30/2 are the graphs associated, respectively, to GL\, O2, 
and SL2. 

In the theorem below, we shall use the following notations. Let n = (ni)jgj, where 
ni € Z> , and E ie /«i = n. Let I(n) := {i G / | m > 0}. Let Y := (g) i6/(lk) Y^ ni . 
For each i E I(n), we shall regard S ni as the subgroup of S n which permutes the factors 
in Y® ni in Y. Let 5„ := n«e/(n) C S n . Let W% be an irreducible representation of 
S nv and let W := <S>iei{n) Thus, the group S R X T n acts on W Y (where T n acts 
trivially on W). Denote by W (g) Y | the (irreducible) representation of G induced from 
the representation W <8> Y of its subgroup Sn x T n . 

We also let 

k(x, y) = uj(x, y)5i + - g)x, (1 - g)y)(c + fcA), 

where c € C(r). We know that this gives rise to the most general continuous symplectic 
reflection algebra (with t = 1). 

Theorem 6.5. (i) Assume k ^ 0. T/ie representation W ® Y" f 0/ G extends to a repre- 
sentation of H K (G) where elements ofV act by 0, if and only if the following conditions 
are satisfied: 
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(1) For all i € I(n), the irreducible representation W{ of S ni has rectangular Young 
diagram, of size ai x b{ . 

(2) For all i,j £ I(n) (where i ^ j), the vertices i and j in the graph associated to V 
are not adjacent. 

(3) For each i € I(n), one has 

dimy, + 2k{b t - <n) + {c,xyM det(l - g\ L )) = 0, 

where XY t is the character ofY{. 

(ii) If E is a finite dimensional representation of H K (G) which is irreducible as a G- 
module, then V acts by zero in E; in particular, E is as specified in (i). 

The proof of Theorem 16. 51 is similar to the proof of jMJ Theorem 3.1, Theorem 4.1]. 
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